Reflections on glossy objects contain valuable and hidden information about the surrounding environment. By converting these objects into cameras, we can unlock exciting applications, including imaging beyond the camera's field-of-view and from seemingly impossible vantage points, e.g. from reflections on the human eye. However, this task is challenging because reflections depend jointly on object geometry, material properties, the 3D environment, and the observer viewing direction. Our approach converts glossy objects with unknown geometry into radiance-field cameras to image the world from the object's perspective. Our key insight is to convert the object surface into a virtual sensor that captures cast reflections as a 2D projection of the 5D environment radiance field visible to the object. We show that recovering the environment radiance fields enables depth and radiance estimation from the object to its surroundings in addition to beyond field-of-view novel-view synthesis, i.e. rendering of novel views that are only directly-visible to the glossy object present in the scene, but not the observer. Moreover, using the radiance field we can image around occluders caused by close-by objects in the scene. Our method is trained end-to-end on multi-view images of the object and jointly estimates object geometry, diffuse radiance, and the 5D environment radiance field.

我们提出了一种学习神经阴影领域的方法，这些方法是神经场景表示，仅从场景中的阴影中学到。虽然传统的形状 - 从阴影（SFS）算法从阴影重建几何形状，但他们采用固定的扫描设置，无法推广到复杂的场景。另一方面，神经渲染算法依赖于RGB图像之间的光度一致性，但在很大程度上忽略了物理线索，例如阴影，这些暗示已被证明提供了有关场景的宝贵信息。我们观察到，阴影是一种强大的提示，可以限制神经场景表示以学习SF，甚至超越nerf来重建其他隐藏的几何形状。我们提出了一种以图形为灵感的可区分方法，以通过体积渲染来渲染准确的阴影，预测可以将其与地面真相阴影相提并论的阴影图。即使只有二进制阴影图，我们也表明神经渲染可以定位对象并估算粗几何形状。我们的方法表明，图像中的稀疏提示可用于使用可区分的体积渲染来估计几何形状。此外，我们的框架是高度概括的，可以与现有的3D重建技术一起工作，否则仅使用光度一致性。